importing hugging face model
Importing Hugging Face models into Spark NLP
Let's suppose I have taken a look at Hugging Face Models Hub (https://huggingface.co/models), and I have detected 7 models I want to import into Spark NLP, for BertForSequenceClassification: Since the steps are more or less the same as described in first example, I'm going to automatize in a loop all the steps from downloading, to importing into SparkNLP and inferring, to illustrate an end-to-end import. There is one extra step we need to carry out when importing Classifiers from Hugging Face: we need a labels.txt, That file can be created using the config.json However, we may find models without that field, what leads to using just numeric values for the labels, what is not very user friendly. To support both importing from config.json and creating our own labels, let's declare an array: If the value is None, then we will import the tags from the model.